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ABSTRACT 

This paper describes two case studies of testing 
programs at local levels. The work was conducted as part of the 
Assessment Development and Use component of the Northwest Regional 
Educational Laboratory's (NNREL) Assessment and Development project. 
Two school districts were recruited to participate in this 
effort — McMinnville, Oregon, and Kyrene, Arizona. The draft case 
studies covered the demographics of each site, the testing system in 
place, the overall history of the activities, and the current status 
of each effort. Comparisons were m:;de on the following subjects: (1) 
costs; (2) change agents; (3) role of technology; (4) curriculum 
alignment; (5) major issues and concerns; and (6) process of change. 
Extensive appendices attached to each case study provide background 
materials, such as five year curriculum plans, testing schedules, 
class profiles, philosophy of education, courses of study, program 
evaluation plans, and cost options. (LMO) 
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OVERVIEW OP THE CASE STUDIES 



The Assessment Development ind Use component of the NIE work of the 
Assessment and Development ^ ' ^t at NWBEL includes two case studies of local 
test developmenc. The purpos. ^as to explore the issues and concerns involved 
at the local level when undertaking such projects. Such exploration included 
examination of such things as why districts chose the testing system they did, 
who the principle players were, what decisions must be made, who needed to act 
as a change agent, how the content for the tests were decided on, and how 
technology was used. Since many school districts are now developing their own 
testing systems in response to the current emphasis on minimum competencies 
and school effectiveness, these cane studies might assist developers to know 
what issues and concerns will arise, the cose of such efforts, and what others 
have decided to do. The case study effort builds on work previously done by 
the project in the area of selecting testing options and using item banks. 

Two school districts were recruited to participate in this effort — 
NcNinnville, in Nor thms tern Oregon near Portland, and Kyrene, in & thcentral 
Arizona near Phoenix. Both school districts have been involved in developing 
testing systems during school years 1984-85 and 1985-86. 

NWREL staff were involved with each district in a number of ways. First, 
project staff attended a number of meetings at each site in which district 
staff discussed plans, issues, concerns and alternatives. Second, each site 
provided a liaison person (or persons) to assist with writing the history of 
the local effort. (Both of these endeavors have roots extending back several 
years. One important aspect of the study became, therefora, tracing previous 
decisions and activities to show the basis for current activities.) Third, 
discussions and interviews occurred between project t,ta^f and important 
players at each site to explore the reasons for decisions, important 
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considerations, issues, problens and concerns. Fourth, NHREL staff provided 
some consultation to the districts to help them find test items and software, 
to train staff on test development and to provide advice on alternatives. 

The draft case studies which are attached cover the demographics of each 
site, the testing system which is in place, the overall history of the 
activities, and the current status of each effort. 

Based on the case studies, the following overall statements can be made: 

1. Costs . Local test development activities are expensive and tine 
consuming. To support the whole curriculum alignment process 
(including test development, curriculum development, and 
cross-referencing), one of the two sites estimates it has spent 
almost $300,000 in the last four years to support its efforts to 
date. The other district estimates about 72 staff release days for 
the development of a single set of K-6 tests on one subject, plus an 
initial investment of $5,000 in computers and software. (Other costs 
were not estimates? by the second district.) 

2. Change Agents . Both districts reported that a high level person in 
the district administration was the Impetus for the project and was 
required throughout the project to organize resources, provide 
motivation, corrdinate efforts, and provide formal directives as 
needed. The specific testing approach that each district took 
depended mainly on decisions made by high level district personnel 
and consultants (with review and approval by teachers and the school 
board). These decisions were supported by varying numbers of 
lower- leveL personnel who "bought into" the approach. Some decisions 
on the specifics involved in the approach were made by the teacher 
committees which did most of the work. 



3, Role of Technology , There waa no question that decisions made by the 
two districts concerning their testing schemes were directly affected 
by the hardware they already had on hand (or had the money to buy) 
and the availability of prewritten items and prepackaged software. 
One district was very explicit that they would not pursue a 
computer- supported testing system unless software could be installed 
on their machines vith little required additional programming. The 
other district made its pattern of decisions based on easily 
obtainable itemp and local software consultants. 



4. Curriculum Alignment . It is equally as clear that current federal 
reports on the status of education in the United States, combined 
with recent research on effective schools, influenced the districts* 
targeting of resources on curriculum alignment. Within their 
resources r both districts attempted to develop criterion-referenced 
survey tests tied to revised curricula to be given in the fall and 
spring to assess teacher and district success in meeting achievement 
goals. Both districts also are attempting to inclement mastery 
learning-type approaches by supporting skill assessments during the 
school year and training teachers in a mastery approach to learning. 



5, Maj or Issues and Concerns > In both districts common issues and 

concerns arose — fear of using the results in a punishing rather than 
supportive way, the time involved in tecting (especially mid-year 
skill tests), the quality of the tests produced, the usefulness of 
the resulting information, fear of over standardization of teaching, 
and the concern that the tests could not measure all important skills 
and so had limited use for diagnosis. One district also indicate^ 
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that the appearance of change by itself was threatening* Thus, 
efforts were made to phase things in slowly, have lots of training, 
do a good deal of PR work, and reduce the appearance of change 
whenever possible. 

Process of Change , Although both districts could develop a set of 
tests in a year, the process ot curriculum alignment, refinement of 
the system (curriculum, hardware, software, and tests), and proper 
use of results, is a multi-year process. There are numerous related 
efforts involved. For example, one of the districts tied the 
curriculum alignment effort into staff training, hiring practices, 
communication with parents, and a career ladder for teachers. In 
fact, both districts see continual refinement and extension of their 
testing schemes at least over the next few years or even cyclic and 
ongoing revision. 



KYRENE SCBOOL DISTRICT 



District Deaoqraphica 

Kyrene is located in southcentral Arizona near Phoenix. The district 
serves about 5000 students in grades It serves as a feeder district to 

the Tenpe Union High School District, The district currently (1985-86) has 
six elementary schools and one junior high school. About 265 teachers are 
responsible for instruction. The district is a rapidly growing suburb of 
Phoenix. Students are generally lower niddle class and predominately white, 
although small communities of Hispanics and Indians are present. 

Description of the Current Testing System 

Testing . Kyrene District has been developing a detailed scope and 
sequence of objectives for all curriculum areas beginning in school year 
1983-84. Currently, these are completed in math and cocmnunication arts. They 
have developed one form of a "survey" test for each completed curriculum area 
in each grade level. The math test was pilot tested in 1983 ^nd became 
available for use in 1984. Cooaunication arts was developed 1983-1985. Test? 
in other curriculum areas are currently under development. (See Appendix A 
for a timeline.; 

These survey tests have four multiple-choice test items to test each of 
30- 60 percent of the objectives at each grade level in each subject. 
Sampling of objectives was chosen to decrease test length. About 90 percent 
3f the items on each test are commercially produced. (The district initially 
used Merrill's custom testing service to test math, but has recently purchased 
their items. The district keeps these items in hardcopy form.) The scope and 
sequence of objectives to be tested and the test items were selected by 
teacher committees headed by an outside consultant. The majority of the items 



on the survey tests are in multiple-choice format. However, in the primary 
grades a portion requires oral responses and teacher observation. In 
addition, a writing sanple is collected at each grade level to measure written 
coomunicatioi. objectives. 

Tests are given in the fall and spring of each year. Students are tested 
on the skills on which they are working. This means that some students are 
tested "out-of-level." The out-of- level procedure was implemented in Pall of 
1985. Criteria for deciding who gets out-of-level tests and the logistics for 
handling these cases are snown in Appendix B. The results are used for 
program evaluation, survey level diagnosis and retention/promotion of students 
at the district level. Teachers are also encouraged to use the results for 
instructional planning. 

Scoring and Reporting . Since there is no in-house scoring facility, the 
district has contracted with a local consultant for development of scoring and 
reporting programs and production runs of answer sheets. The reports 
available through this system are shown in Appendix C, Briefly, these reports 
are: 

^« Class Profile Report . This report includes a summary for each 

classroom of skills mastered, and a summary for each student of 

skills mastered and local percentiles. 
2. Individual Student Profile . This report lists, for each student, the 

individual objectives mastered and not mastered, percent of 

objectives mastered and the local percentile. 
3* School Summary Report . This report provides a summary of mastery and 

local percentile information for each class and each grade in a 

school . 

4. District Summary Report . This report suimnarizes grade level and 
school information across the district. 
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Regrouping Report , This provides objectives mastery class profiles 
for all students mastering less than 40 percent of the grade-level 
objectives regrouped by next year's teacher. 
The cost of the scoring and reporting functions is currently about $28,000 
per year. This includes scoring a total of about 20,000 answer sheets (pre 
and posttest in all grades and two subject areas) , production of all reports, 
provision of preslugged, general-purpose NCS answer sheets, and instructions 
for administering and returning the tests. Kyrene district has the 
responsiblity of supplying scoring keys and to reproduce and distribute tests. 

Related Efforts . The first effort that is related to Kyrene* s criterion 
referenced testing program is teacher training. All new teachers receive 
about three hours of in-service on the logistics and philosophy of the testing 
scheme. Training on logistics for returning teachers is not deemed to be 
necessary. 

Training on the use of results (e.g., school board, principals) occurs 
during regular interactions on the topics. For example, presentations to the 
Governing Board on results is accompanied by information about philosophy and 
purposes. Training to teachers working on coomittees occurs when they get to 
a point where extra information is needed. (More detailed training on use of 
results for planning instruction is beginning this year and will be descr^.'^ed 
later in this paper.) 

Another related effort involves hiring practices. Part of the criteria 
for hiring teachers relates to their philosophy concerning and ability to use 
a mastery learning approach to teaching. 

A final related effort involves the newsletter that Kyrene usually sends 
to staff and parents. The emphasis in the newsletter is instruction and 
regular articles appear on achievement results and current activities on the 
scope and sequence and test development. 
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Summary , Thus, as of the Fall 1985, the district has its curriculum 
developed in math and communication arts, has developed and used locally 
developed tests in these areas, utilizes outs.de customized scoring and 
reporting, and conducts voluntary teacher training on the use of test results. 

Hiatory of Implementing the Testing System 

Initial Impetuses . In 1972 the State of Arizona mandated that all 
districts muld develop a "continuous and uniform evaxuaticn system" (CUES) to 
establish performance targets for students at each grade level. The 
superintendent at Kyrene wanted to use this mandate to develop a meaningful 
critrr ion- referenced testing system and developed a five-year plan to develop 
such a system. (The five-year plan is in Appendix A.) The five-year plan was 
based on and an extension of current Kyrene educational philosophy and 
curriculum development and adoption procedures (see Appendix D) . The 
five-year plan was intended to be cyclic and was based on the state's textbook 
adoption cycle. 

In 1980 the district received a federal grant for approximately $100,000 
to develop such a criterion- referenced testing system. Tnis system was in 
place by 1980-81 at a cotal cost of $200,000. Essentially the original 
testing system involved specifying a minimum set of skills to be attained at 
each grade level and developing tests to be given in the fall and spring to 
see now well these skills had been learned. In addition, teachers were to map 
skills continuously throughout the school year. The district reports that 
this original product has been adopted by many districts nation-wide. 

There were several problems that emerged with this first scheme. First, 
many students had already mastered most of the tested skills at pretest time 
which made the tests less than ideal for instructional planning. The tests 
were essentially minimum competency tests. Second, there was a general lack 
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of proper use of the system by teachers. For example, many teachers waited 
until just before they were monitored to update mid-year skills lists. Third, 
teachers complained of too much recordkeeping. Fourth, it was not based on 
any real scope and sequence, but only on target skills for each grade, so it 
was difficult to use for instructional planning. Fourth, there were problems 
in the logistics of testing and recordkeeping. 

In 1981-82 a new Assistant Superintendent for Instruction was hired who 
was knowledgeable in the area of effective schools research. One reason she 
was hired was to make adjustments in the criterion- referenced testing program 
to solve seme of the problems which had been encountered. After attempting to 
tinker with the current system for two years, the decision was made to phase 
in a new testing scheme based on a revised scope and sequence. 

In March 1983, the Governing Board passed a policy to support the new 
effort. This policy (see Appendix E) advocated a continuous scope and 
sequence to: 

1. promote continuity across grades, 

2. make it clear to teachers, parents and students what was expected in 
terms of skills acquisition, and 

3. align the curriculum with testing so tiiat progress in the skills 
areas could be followed. 

The policy also stated tnat minimums and "extended learning outcomes" 
would be provided, it provided for survey-type fall skill preassessment tests 
to be developed in order to place each student in the curriculum, it called 
for formative assessment to measure studenc progress and form the basis for 
promotion, and it required appropriate recordkeeping to track student skill 
levels. The Assistant Superintendent for Instruction was placed in charge of 
directing the planning, implementation and evaluation of the system. Efforts 
were to be made to coordinate the process with the Tanpe Elementary and Tempe 
^ Union High School Districts. 
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Thus, the impetus for the current test development effort came from 
several sources: 

1. A state mandate for CUES 

2. A superintendent ..crong in instruction who wanted to respond to the 
mandate in a meaningful way 

3. A desire to redo a previous effort to mr ? it more useful for 
diagnosis and instructional planning 

4. An Assistant Superintendent for Instruction which had a clear idea of 
what a good criterion-referenced testing system should look like*. 

5. The availability to the district of money to pursue such an effort, 
first in the form of a federal grant, and more recently, in the form 
of "sudden growth" money because this district is experiencing a 
current jump in population. 

In order to implement the new policy, the Assistant Superintendent for 
InstrucTtion and an outside consultant developed a program eva^' ^.ion plan 
which updated the previous five-year plan and called for curriculum 
development, test development and program evaluation. (Outside consultant? 
began to be used because of two district philosophies. First, one reason that 
the previous sytem failed was that it was developed entirely in-house without 
benefit of outside expertise. Second, the district does not feel that one 
needs to develop ownership by doing everything oneself. It can be equally 
effective, and much more efficient, to react to products developed by 
others.) The cycle was to begin in school year 1983-84. (See Appendix F for 
the evaluation plan.) This plan provided criteria for cyclic review of the 
curriculum. The criteria included 'development ... of an evaluation tool to 
be used to measure student achievement of a sample of curriculum objectives.* 
After the tests are developed, b^sic skills areas were to be tested annually 
and other areas periodically. In order to judge program effectiveness, 
O 11 
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criteria wete established to student performance on the tests. If studencs 
did not perl'oru up to the criteria, provisions were ude for the evaluation of 
instruction. These plans were then reviewed by staff and adopted by the 
Governing Board. 

Two interesting points are, first, that the paporwork to support the 
effort was developed at different times by different people and was brought 
together in a coherent whole at the beginning of the current effort, Seconi, 
the old testing scheme was, and is, being used until the new curriculum and 
tests are phased in as part of the cyclic process. 

Curriculutt and Test development . Development of the new scope and 
sequences began in Sunmer, 1983. The district proceeded by hiring a 
consultant to do a literature search to see what others had done. The 
district judged that most other systems also tended to have skills targeted at 
ainimuffls and so were not useful for diagnosis and instructional planning. 
Thus, they decided to develop their own scope and sequence f» and tests. 
Ccnmittees of teachers were established headed by an outside consultant. The 
outside consultant developed the initial smpe and sequences. The committees 
validated the scope and sequences, selected the skills to be tested and 
acquired test items identified by the consultant. These products were then 
reviewed by all staff and approved by the Governing Board. 

Since the skills lists were very detailed, not all could be tested in a 
survey test. The teachers choose from 30 to 60 percent of >he objectives to 
be tested. To avoid the need for local development of ii^ems, the district 
initially used Merrill" a item bank. The decisis to u^^e previously developed 
items came from past experience with the poor quality of items developed by 
local personnel. 
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The math tests were developed first. These were pilot tested in school 
year 1983-84. The main result of this pilot testing was to move many skills 
to an earlier grade in the scope and sequence, attd revise the tests 
accordingly. These changes were made because students seemed to acquire 
skills earlier than was thought. The pilot test also resulted in a few items 
being revised or replaced. Revision of it«ics was limited because good quality 
items were aleady being used and because of the extra expense involved in 
obtaining item statistics from the outside scorer. 

Development of Test Scoring and Reporting . The district initially used 
Merrill's custom testing service to develop their math tests, score them and 
produce reports. Because of the expense involved in this and because of the 
continual addition of new tests, during school year 1983-84 the district 
contracted with a local consultant to provide scoring and reporting services. 
This consultant had also scored the old tests and so phased the new tests into 
the existing scheme. During this time report formats were modified a few 
tim«^s based on input from teachers and other staff. These changes were mainly 
cosmetic because. the teachers like Moirill's presentation of results better 
than that from '*he consultant 

Teacher Concerns , Thet .one initial concern by teachers about this 

etfort bec7.se of their recent experience with a testing scheme which failed. 
Many teachers had develop i ownership of the old scheme. Others felt that the 
Idilure meant that such efforts were futile. Also, tnere was concern because 
of change itsexf — there had been ongoing changes in logistics and report 
format and this represented anotht.^* set of changes. 

Other teacher concerns developed during the implementation of the testing 
schone. Since the skills were made harder, many more students do poorly on 
the pretests in the fall. SCTie teachers felt thaw these tests are too 
traumatic for the students. (Pretests may be dropped at kindergarten and 
Q grade 1 for this reason.) 
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Co8t£. As Of Pall 1985, the district estimates it has spent about $70,000 
per year for the last four years to inplement this process. Products thus far 
include tests in two curriculum areas and revised scope and sequences in one 
curriculum area, and purchase of 6,000 test items. Costs include staff tine, 
consultant tine, printing, and purchase of items. The scoring and reporting 
costs an additional $28,000 a year. Since this process is cyclic, the 
district estinates ongoing costs to be about $50,000. (This may change if 
some of the refinements being currently considered are implemented.) 

Current Status 

Por the most part, the district now feels that the major conceptual 
fraroewrk of the system is complete and is reasonably satisfied with the level 
of difficulty of skills represented in the scope and sequences and the tests. 
Because of the amount of change which has occurred recently, the district has 
decided to let the current tests in the mathematics area and logistics stand 
for the next two years. Kajor current work will be in the areas of efforts 
related to the testing scheme, development of the scope and sequences and 
tests scheduled for this year by the five year plan, oollectioi of information 
on tests and items which will be used in the future to refine both the tests 
and the levels at which skills will be targeted, and laying the groundwork for 
changes in logistics in two years. 

Related Efforts . The district now feels that F^ost effort should be put 
into training teachers and principals concerning the proper use of the 
system, rhe district estimates that about 40 percent of the teachers 
currently use the testing scheme to its full advantage. Many of these 
teachers are in schools which have a principal which encourages its use. 

Teacher Training . Staff development on effective teaching began in 
1981-82. The training program was based on the effective teaching literature, 
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especially the B'JIS sodel (Berliner and Rosenshine, 19 ). Sessions were 

offered for district and/or ASU credit. The first emphasis in this training 
was on clascroon management and other topics besides recordkeeping. This was 
a conscious decision because many of the teachers had negative pest 
experiences with effective teaching being only related to keeping records. 
The component of the training to begin this school year (1985-86) relates to 
the development and use of criterion- referenced tests to monicor student 
progress and modify instruction. The new scopes and sequences and tests will 
be part of this training to encourage prc^r use of test results. Another 
thrust will be to enable teachers to test and monitor district skills in the 
scope and sequence not on thu survey tests. 

2. Career Ladder . The district will begin this year to implement teacher 
advancement based on increasing skills expected as a teacher. This ties into 
the testing program which will be used to monitor outcomes and teacher 
training and e^raluation which emphasizes effective teaching skills. 

3. Teacher Evaluation . Teacher evaluation on the skills demonstrated by 
effective teacher a as shown by research will be inplementod this school year. 
Monitoring student progress and using information to guide instruction will be 
part of this process. 

Logistics . One big are^* in which the district has been considering change 
is in logistics. Based on the first two years (1983-1985) of testing 
information, outside contracting and local test development several needs and 
issues came to light. 

1. It was very expensive to utilize outside scoring. The best 
turnaround time obtained tfas two weeks. The district began 
considering scoring tests in-house in order to decrease costs and 
improve turnaround. 
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2. It was very expensive to develop tests from the items housed at 
Merrill, but there was a need to develop alternative rorms and revise 
the tests based on curriculum changes from the pilot years. (The 
district bought the items in June, 1985.) 

3. Only a portion of the skills in the scope and sequence were tested by 
the survey tests. Thus there was no formal mechanism to aisess 
student skills in the other areas. This function was left up to 
teachers. Teachers expressed an interest in a bank of items they 
could use to develop tests for diagnostic and mastery purposes. 

4. It was difficult to keep track of student skills. Such a tracking 
appioach to student skills can require a lot of recordkeeping. Thus 
far the district has pre* and posttest ccnparisons of skill levels, 
but no ongoing method of keeping track of individual students. 

These needs, plus the continued availability of "sudden growth" money 
suggested local computerization of test development, scoring and reporting. 
This became feasible because, at the same time, the district was deciding on a 
minicomputer to use for student attendance and business office functions. 
(This was also being purchased because of the "sudden growth" moneyO The 
district hired a consultant in the Spring of 1984 to study district computer 
needs and match these to hardware and software options. The HP3000 wae 
selected in Pall, 1984. Pertaine was awarded the contract for training and 
service. Part of the reason for choosing this hardware and software was that 
Portaine was also vending an instructional management, test scoring and item 
banking system developed by Adaias County, Colorado. 

In March, 1985 the district seriously began to consider the Adams County 
software (called the CMI3000). A series of meetings took place bet.^een the 
consultant hired to develop the curriculum and testing scheme, the Assistant 
Superintendent for Instruction, one principal particularly interested in 
O 16 
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computer uses, the Superintendent, the business Manager and the director of 
special services 'e.g., special education). These people (plus consultants 
hired by the district) constituted the •Instructional Ccoputer Connnittee" 
which was asked tot 

1. define the essential issues and gustions related to acquiring a 
testing and reporting system; 

2. examine and report on the currently available software systems 
compatible with the district HP300; 

3. examine and report on the potential costs of acquiring a system for 
in- house scoring and reporting; and 

4. specify both the short-term and long-term tasks necessary to develop 
and acquire a system should one prove to be available and 
cost-effective. 

Because of perceived lack of tiiuv^ and expertise to pursue some of these 
issues, an outside consultant was hired in April 1985 to assist with the 
computer portion of the task in April, 1985. 

Because the district was considering doing item banking, they contacted 
NWPEL al'ter one of the consultants read Arter and Millman's article in JSM . 
NWREL met with district staff four times to discuss alternatives, look at 
considerations and identify alternative software. In return, the district 
agreed to be a case study site. 

During these meetings, the following were identified as iraportant design 
features of the system: 

1. There needed to be quick turnaround since teachers wanted pretest of 
group skills early in the school year to plan instruction. 

2. Reports needed to be easy to read and change as little as possible 
from the ones currently in use. This was seen as important because 
the teachers had been through so many testing changes in th^ past 
three years. 
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3. The district wanted a system that would produce achievement 
comparisons between schools and teachers, and would provide item 
analysis on tests. 

4. Two independent purposes for testing were the district level "survey" 
tests and the interim tests to be developed by teachers. There was a 
need for security fot the test items to be used on survey tests. 

5. Since the district has no permanent programmer # software needed to be 
purchased "ready made." 

6. The district wanted to acquire items which were already pllot-testisd 
in order to minimize the need for local development and maximize the 
quality of the tests. 

7. Special education wanted lEP's automatically generated. This would 
necessitate a cross-reference of skills to materials. 

8. Teachers wanted open access to items, the ability to add their own 
items, and autcnatic report catds. The teachers did not have so much 
concern for curriculum and program evaluation, but rather wanted 
information for planning instruction. 

9. There was a software agreement with the vendor supplying software for 
other district uses of the HP3000. If the program is not compatible 
with current systems then the maintainence agreraent is void. 

10. The HP3000 might not have enough disk space to run an item-banking 
program. 

11. The HP3000 was initially purchased for administrative uses such as 
payroll and student attendence. There would have to be policy set 
regarding the availability of the computer for testing. 

12. The current softwa^^e vendor is a business i^ich does not particularly 
know about educational concerns and issues. The district felt that 
it was on its own to troubleshoot. 

13. The district felt that it needed an evaluaticxi unit to handle this 
and other functions %#hich are currently spread out in the district. 

14. Outside consultants are needed to asist with the plans and also to 
provide credibility to tlie process. 



The district has tentatively decided to purchase the Adams County software 
in one to two years. Three other systems were identifiei which run on a 
HP3000 computer. The others, however, are not in disseminatable form at this 
time. The district has no permanent ccnputer person and so wanted a package 
that could b€ used "as is" with little additional programming, nwrel also 
helped the district assess this software package and found it to be reasonably 
^ flexible for the tasks desired. 
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The Adams County software cones with several files whicn are 
cross-referenced to each other: 

1. Instructional objectives 

2. Instructional resources 

3. Test items 

4. Student information 

5. Tests 

Survey tests are already developed and stored. The software scores tests has 
17 different report features and automatically updates the student information 
file. The user has the option to use all prepackaged features or use the 
software to enter and use one's own objectives, items, materials and tests. 

In the end, the most ii^rtant considerations in deciding on how to handle 
testing logistics turned out to be that the software was prepackaged and 
different parts could be phased in as time went on. It has the potential for 
teacher access and use of a large number of items, provides scoring and 
reporting, and it was vended by the same company which provides other district 
software (and so would not void the previous maintainence agreement) . The 
overall cost to the district does not seem to be the overriding concern as can 
be seen by cost projections in Appendix G. It was also decided that the need 
for match-up to current report formats was a secondary concern. 

The current plan, which will be refined over the next year or so will 
entail developing the scoring, reporting and lecordkeeping features first. 
The district will enter its own skills. After an examination of the skills 
lists provided with the Adams Counf;y package, the district found their own to 
be more comprehensive and levelled better. Since the district already has 
tests in place which measure these skills, they will cross-reference tests and 
items to the skills entered on the system. When tests are scored the computer 
will automatically update the student record file. 




Once the scoring and reporting features are in place, the district might 
consider (1) developing new test forms and (2) item banking for teacher use. 
The former plan would entail keeping the Meirill items in hard copy format and 
restrict them to district survey use. The formal district survey tests would 
be updated by a teacher coiunittee guided by an outside measurement person 
every two to three years. The latter might enUil using the Adams County 
items which are already on the system and cross-referencing them to Kyrene*s 
skills. Boles would be filled n with items from other sources. 

Once these are in place th^ district mignt (1) acquire other items and 
(2) pursue using die system for other types of program evaluation. The latter 
would entail some programming in order to send testing results generated from 
the system to an HP3000 system file which could then be merged with other 
HP3000 system fil«s and accessed by a statistical package. 

The computer consultant is preparing an Implementation plan whici includes: 

1. Changes which might need to be made in the district management 
structure in order to accomodate the testing configuration. 

2. The steps involved in purchasing, installing and maintaining the 
software. 

3. The steps needed to develop tests frcn the system, including the 
entry of skills and items onto the system. 

4. Activities needed to provide timely reports on student skills. 

5. St^ff development. 

6. The expandability of the system to include future functions. 
(See Appendix A for th? most recent version of the implementation plan.) 
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FILE ICR-E 



EDUCATIONAL PROGRAM DEVELOPMENT 

FIVE YEAR PLAN 
FOR 
CURRICULUM 



Planning 



Design 



Try-out/Program 
Evaluation /Design 
Revision/ Implement 



Year 1 



Year 2 



Years 3-5 



83-8A Computer Sci, 
Reading 
Li terature 
Vocabulary 
Oral Comm. 



Written Communication Mathematics 

Grammar Health Education 

Spelling Social Studies 

Study Reference Skills Home Economics 



Sc ience 



Industrial Arts 



8^-85 Library Sci. 

Foreign Lang, 
Handwriting 



Compu ter Sc ience 

Reading 

Li terature 

Vocabulary 

Oral Communication 



Written Communication 

GrammPi' 

Spelling 

Study lief. Skills 
Sc ience 



85-86 Career Ed. 

Physical Ed. 

Music 

Art 



86-87 Mathematics 
Health Ed. 
Soc . Studies 
Home Ec . 
Ind. Arts 



Library Science 
Foreign Language 
Handwriting 



Career Education 
Physical Education 
Music 
Art 



Computer Science 

Reading 

Li terature 

Vocabulary 

Oral Communication 

Library Science 
Foreign Language 
Handwri ting 



87-88 Written Comm, 
Grammar 
Spei: ing 
Study Ref. 

Skills 
Science 



Mathematics 
Health Education 
Social Studies 
Home Economics 

Industrial Arts 



Career Education 
Physical Education 
Music 
Art 
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MSS TESTING 



FALL 



SPRING 



LEVEL 



UHO GIVES 



WHO GETS 



On Grtde 
Level Testing 


Classrooa Teachers 
Elementary 4 lunlor 
High 

(Fall 1 Spring) 

Special Ed. Teachers 

(Spring) 


AU Students EXCEPT 
e Special Ed. Studenta 

(2 yrt below gr. level) 
e Girted Studenta 

(who took teat in the 

Spring or •Bk and achieved 

90% or greater) 


During Fsll 
Testing Parlod 


• All students In District 

(Includlnc SMclal VA 

and po^alble retalneea) 


e Early Spring 

i9Sb pvrioo 1 Or 

Special Ed and 
potential eleaentery 
retalneea 
e All others • Spring 
Teating period 
(Including Jr. High 
retalneea) 


Above Grade 
Level Testing 


Eleflentary Girted 
Teachers 

Junior High Assigned 
Hath or Honors 
Teachers 


e Identiried Girted Studenta 
OR 

e Studenta Msterlng > 70t 
or grade level obJectlv«9 
n Students In rererrsl process 


Arter Receipt or 
Pall Grade Uvel 
Test Reports 

(October) 


e Identiried girted atudenta 
In Hath 

e Any atudent receiving Mth 
Instruction sbove grade level 
as a result or Psll pre-testlng 


Spring Testify 

period 


Below Grade 
Level Testing 


Speclsl Educstlon 
Teschers 


e Identiried Special Bducstlon 
StudenU In Hath EXCEPT those 
students who took test In 
Spring or "B^ 


During Pall 
Testing Period 


e Identiried Special td. in Math 
and received Instruction on 
below-grade itvti objectives 

• Possible retslneea 
(give grade level and one year 
below grade level teat) 


Early Spring 
Teat period 

• 


As assigned by 
principal 


e All students new to Kyrene 
District receive 1 grade 
level below assigned grade 
level test 


Arter receipt or 
results or Fall 
Testing Period 
OR 

As student enrolls 




Special Education 


e Students In rererrsl 
process 


Upon reFerence 







• In early aprlng, Special Education and poaalble retalneea will take two teata 
on grade level and below grade level. 
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CASS TESTING 



SPRING 



LEVEL 


WHO GIVES 


1 

MHO GETS 


WHEN 


On Grade 

L( Testing 


Special Education 
Teaphers 


Potential retalnees 
Special Ed. Students 


Early Spring Test 
Period 




Classroom Teachers 


All 5tudents EXCEPT 

• Potential Retalnees 

• Special Ed« Students 


Spring Test Period 


Off Grade 
Level Testing 


NOT TO BE GIVEN DURING 
PILOT YEARS 
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eUUCATtONAL CVALUATION SYSTCHS, INC. 
DATE - OS/3 1 /an TEST - HATH 0 
8TUMIIT8 TCSTCD - 
STUDeNT NAHB 



DETAIL CLASS PROPILC 
T6 ACHES - nCLAUQHLlN EOSE 



OEADE - S 



HASTESY LEVEL 
ITEHS TESTEO 



1 2 3 . , • 7 . . i I iTtViTn^IVn"? i I n i s ? ! 1 2 1 i 
2 u 3 s 2 s s s s s 2 2 s ; s s 2 s 1 1 1 1 1 1 1 1 1 1 1 




Incisco 

jjTBVEIi 

jacSUeline 



RS HASY LOU 



PERCENT OP STUDENT 
MASTERY - CLASS 



KS5III? • 



gERCEJT OP STVPe 



STRICT 



2 i 




IM\\ 

i H u i f i i i i i 1 1 u . u u u I . u i n } > . . u 

U^UUUl'hitUlUliiitii.lSiiiiit 



^ • 

0 7 4 



m 



n t H ? : ? u I ! u 2 ! . ! n 1 n 

SKILL NUmCR AND NAM 
10 Ntm 



SUN VALLIV USD OliTltCT 
THONAS iOISON SCHOOL 
TisT 



OORRCCT 
TOT % 



4 - 



9i 



SKILLS f 

lis 



» 



SKILLS 

li 

74 

t\ 

SO 
S2 

\l 

9 

Is 

AVERAGE 94 47 22 40 
• 27X STUOESn IR .LOV 

^sep 



If u 



h 



BASED ON SKIL 



fa 
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EDUCATIONAL EVALUATION SYSTEMS 



SUM VAii.rY usn 



srMf-inL, THQitAS >ni<nii 



^>y. MATH fl 



LM 




TOTL 



ntno 


irvM 




SKILLS 
HAST£RED 






NUMtCN 


mccMT 










100 


So.o 


. S6«0 


11 
25 




01 .d 
92. 0 




11141 


W A 
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EDUCATSONM. IVALUATlON tmCNt* INC. 
DUTC - tS/l^/M TiST - MTH • 



COmilflCD tCNOOL •vimMY 
TEtT I TENS - 144 MILLS 34 



•UN VM.LfY UtO OltTlllCT 
TNONM lOttON MNOOL 





TCNR 
1 


TCNR 
2 


TCNR 

3 


TCHR 
4 


SCNL 


OlST 


NUNaEII OF STUDENTS TESTED 


ss 


27 


27 


2^ 


1SS 


517 


AVG NUN8ES Of ITEMS CMSECT 


S7 


64 


SS 


SI 


70 


SI 


SVG PERCENT Or ITEN9 COSSECT 


4S«4 


4<.4 


47. f 


S3.2 


84.2 


42.4 


SVG HUNSES Of SKILLS HSSTESEO 


IS 


18 


IS 


22 


18 


13 


SVG PERCENT 0/ SKILLS NHSTERCO 


52. S 


41 .7 


44.4 


SI .1 


81. S 


3S.1 


SVG LOCM. PERCENTILE RSNK -SKL 


72 


83 


8S 


•S 


44 


9S 


PERCENT Of STUDENTS NSSTERINQ 

75X OR NORE OF SKILLS 


3S.3 


27.4 


3S.1 


4S.S 


39.0 


27.8 



DENOGRAPNIC SUKNARY 


CAT 
1 


CAT 
2 


CAT 
3 


CAT 
4 


SCNL 


DIf 


NUN8ER Of STUDENTS TESTED 


5S 


14 


SS 


IS 


IDS 


517 


AVG NUN8ER Of ITENS CORRECT 


rS 


S2 


8S 


•3 


78 


SI 


AVG PERCENT Of ITENS CORRECT 


54. S 


43.1 


4I.S 


87.S 


54.2 


42.4 


AVG NUN8ER OF SKILLS HASTERED 


IS 


18 


14 


2S 


18 


13 


AVG PERCENT Of SKILLS NASTEREO 


52. S 


41. • 


30. S 


55 .4 


8S.S 


3S.1 


PERCENT Of STUDENTS NASTERINQ 
79X OR NORE Of SKILLS 


37. S 


2S.7 


3S.2 


39.3 


39.8 


27.5 



TCACNER NUNSER AND NANE 
I DARLENE JONES 



2 EDCAR HILL 



3 MILLIAN RANDOLf 



4 ALFRED NORCANSTCRN 



DCNOCRAPHIC CATEGORY NUH8CR AND NANE 

I CAUCASIAN 2 BLACK 



3 HISPANIC 



4 ASIATIC/ORIENTAL 



KSTCOPTW/WUBtE 



34 



tOUMTlONM. EVM.UAT10N •VSTCNt, INC. 



COHBIWO MtTRlCT •UNMRY 



MTC - •9/lt/t4 TCtT - MTN t 






TfiSI 


ITEMS 


- 144 


SKILLS 34 




SCHL 

1 


SCNL 
2 


SGML 

3 


SCHL 
4 


SCNL 
9 


DIST 


NUMBER Of STUDENTS TESTED 


1SS 


1S4 


1S2 


94 


1M 


917 


MQ NUftSER OF ITEMS CORREGT 


7S 


71 


99 


93 


44 


41 


RVG PERCENT OP ITEMS CORRECT 


94.2 


91 .1 


40.1 


34. B 


44.4 


42.4 


RVC NUMBER OP SKILLS MISTERID 


ts 


17 


12 


IS 


19 


13 


RVQ PERCENT OP SKILLS MMTERED 


90. • 


47.2 


33.3 


27.7 


41. S 


34.1 


RVQ LOCRL PERCENTILE MHK -SKL 


44 


94 


4S 


49 


92 


91 


PERCENT OP STUDENTS MMTERING 
79X OR MORE OF SKILLS 


39. S 


33.2 


24.3 


24.1 


29.2 


27.9 


DEMOQRRPHIC SUMMARV 


CAT 
1 


CAT 
2 


CAT 
3 


CAT 
4 


DIST 




NUMBER OP STUDENTS TESTED 


330 


47 


78 


99 


917 




RVO NUMBER OP ITEMS CORRECT 


M 


91 


4« 


S7 


41 




RVQ PERCENT OP ITEMS CORRECT 


49.9 


39.4 


33.7 


44.9 


42.4 




AVQ NUMKR OF SKILLS MASTERED 


19 


11 


12 


13 


13 




RVQ PERCENT OF SKILLS MASTERED 


41. S 


3S.i 


33.3 


34 J 


34.1. 




PERCENT OF STUDENTS MASTERING 

7SX OR MORE OF SKILLS 


2B.7 


2S.2 


22.3 


27.1 


27. S 




•CNOOL NUMBER AND NAME 
1 IHOHAS EDISON 


2 STARLIGHT 


PARK 






3 SUNSET HILLS 



SUN VALLEY USD DISTRICT 
GRADE - B 



S SUN VALLEY JR Ml 



4 MOUNTAIN VISTA 



DEMOGRAPHIC CATEGORY NUMBER AND NAME 

1 CAUCASIAN 2 BLACK 



3 HISPANIC 



4 ASIATIC/ORIENTAL 
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FILE lA 
KYRENE SCHCX)L DISTRICT NO. 28 
PHILOSOPHY OF EDUCATION 



Purpose of the Philosophy This philosophy of education has bean 

established by the Governing Board 
of the Kyrene School District to pro- 
vide a broad set of int<.rnally consist- 
ent referrents which guide the activities 
of the members of the educational staff 
of the school district as they plan for 
and implement programs for children and 
youth. 

It shall be the mission of the Kyrene 
School District to provide experiences 
which facilitate the growth of each 
student that he/she may lead a life 
which is personally satisfying and 
which contributes to the society which 
sustains him/her. 

The broad, overarching goals of the 
Kyrene School District are: 

1. Development of competence in the 
basic skills in reading, composi- 
tion, listening, speaking, and 
computation. 

2. Development of skills in ways of 
creative and disciplined think- 
ing and application of knowledge. 

3. Development of fundamental under- 
standing of the humanities and 
the arts, the social sciences, 
and the natural sciences (in the 
form of basic concepts and gen- 
eralizations) . 

4. Learning how to learn; how to 
attack new problems; and how to 
acquire new knowledge. 

5. Development of self -understanding; 
self-respect; self-direction; and 
self -instructional skill. 



Mission of the Kyrene School 
District 



^ Goals 
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Kyrene School District 
Philosophy of Education 
Page Two 



Goals r Continued 6. Development of a zest for learn- 

ing r an abiding interest in 
learning • 

7. Development of a sense of social 
responsibility. 

8. Development of skill in the ex- 
ploration and clarification of 
values. 



Interpersonal Relations The bases for all interpersonal re- 

lationships in the Kyrene School 
District shall be a respect for 
human dignity . The very foundations 
of our nation are rooted in this 
basic principle. The intrinsic 
worth and dignity of each individual 
in the school system must be rec- 
ognized and must be kept in mind in 
the planning, implementation r and 
evaluation of the individixal's pro- 
gress and of the overall effective- 
ness of the school system. The core 
of values of a democratic society are 



+ All persons should have worth 
and dignity; a right to respect; 
should never be a means to some- 
one else's ends. 

+ All persons have a right to make 
decisions about matters that 
shape their lives. 

+ All persons have a right to educa- 
tional opportunities appropriate 
to their differences. 



+ Diversity is essential to renewal 
of an open society and a right of 
the individuals within. 

Schooling - Training and In providing schooling r the major 

Education thrusts should be of two types - 

training and education. Training 
is primarily the imparting of basic 
skills, knowledge and attitudes, 
while education is concerned with 
the more sophisticated processes and 
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Kyrene School District 
Philosophy of Education 
Page Three 



Schooling - Training and 
Education r Continued 



concepts which are built from the 
basic skills. Education is a broad- 
ening process: it opens doors and 
makes student's aware that there are 
many doors in their lives to be 
opened, choices to be made as to 
which ones will be explored and made 
a part of their lives. Thus, train- 
ing may be viewed as concerned with 
those skills, facts, and attitudes 
which we deliberately "put into** 
students while education asks the 
questions, "What is there unique 
about each individual which should 
be 'quickened and developed'?" 
Thomas Carlyle said, "The tragedy 
of life is not so much that men suf- 
fer, but rather what they miss." 



Learning Environment The learning environment must be 

characterized by the quality of car- 
ing. Students must feel that concern 
for their well-being is primary and 
that relationships with their peers 
and the staff of the school are 
guided by concern for others. 

In closing, it must be pointed out 
that the referrents in philosophy 
are necessarily broad. This char- 
acteristic is positive in that it 
allows for inclusiveness and flex- 
ibility. The broadness can be 
negative if the Governing Board 
and educational staff do not con- 
tinually seek higher levels of 
understanding of the broad general- 
izations of the philosophy. With- 
out this dedication to understanding, 
a philosophy is meaningless and has 
no value in practice. 
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CURRICULUM DEVELOPHENT AND ADOPTION 



The currier Ixim of a school system is the organized system of 
content and processes on which the instrv^ctional activities for 
students are based. It should reflect best knowledge of the 
growth and development of learners; the content o^ the various 
content disciplines of man; and the needs of Ic^ar rs based on 
the navure of society and the desires of the p/ • ons of the 
District. It is the intent of the Kyrene Schoo^ District to 
develop a set of curriculum documents whicu provides the basis 
for instructional activities and to review and revise the 
documents when appropriate. 

The establishment of a significant curriculum document to guide 
the educational staff in providing learning experiences fo^ 
students shall be a priority ol the Governing Boar^ for the 
Kyrene School District. The Superintendent of Schools has the 
responsibility of as-5uring its completion on schedule. The 
Governing Board herein pledges the provision of reasonable 
resources to support the District's plan to develop curriculum 
documents in accordance with the proposed schedule. 

The basic responsibiiJ ^ for curriculum development shall reside 
with the Assistant Sui intendent for Instruction working with 
principals and teachers His/her responsibilities include: 

— Providing leadership to the schools individually and 
system*wide. 

— Coordinating the planning and the decision-making so 
that a common direction of action is provided for the 
school system. 

— Working with principals and teachers of the individual 
scIaOoIs in designing a curriculum which manifests achieve- 
able challencj^^s for all students. 

— Communicating to the schools information which affects 
system*-wide agreements and plans for curriculum 
development. 

--Communicating with the Temp-* Elementary School District 
and the Tempe Union High School District in articulav,ing 
a curriculum spanning grades K*'12 

Each year, the plan for completing curriculum d«*..j.g s for the 
coming year shall be presented to the Governing Boaru no later 
thdn May 1. Periodic reports will be made to the Board of 
progress during the year of design. The completed document shall 
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be presented to the Board for adoption in accordance with the 
provisions of appropriate Arizona statutes. 

iini-ini.^® policy of the Kyr-ne School District that curriculum 

patterns, and scheduling be in compliance 
with Title IX of Education Amendments of 1982. compiiance 
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\) COURSES OF STUDY AND SUPPORT SYSTEMS 



The Governing Board believes that learning will be enhanced 
by adherence to courses of study which promote continuity 
and cumulative acquisition of skills and knowledge from grade 
to grade ai\d school to school. The courses of study are 
designed to provide teachers and students with the district's 
expectations of what children and young people are to learn. 
Teachers are expected to follow those courses of study appro- 
priate to their teaching assignments. 

The design of the courses of study system will meet the following 
guidelines : 

1. The skills and knowledge to be acquired will be specified 
in the form of learning outcomes to be mastered by students. 
Learning outcomes will be derived from significant content 
and processes. 



2. The learning outcomes will be arranged sequentially so 
that learnings are built one upon the others concurrently 
so that they reinforce one another as appropriate, 

3, The learning ouLcomes will be comprehensive at each grade 
level, providing challenge for all students. They will 
include : 

•1 Foundational learning outcomes which all students 

are expected to learn. The foundational outcomes 

j^^^ will be adapted as needed for individual handicapped 
students. 



.2 A contini;um o' extended learning outcomes through 
which students' will progress to the maximum of their 
IT ability. 

In addition to th;;^ courses of study, appropriate support 
systems will be developed: 

1. A c riterionr referftncec^ aflaeagmen t aystem will be designed 
to assess student progress and for insTFuct ional planning. 

C.l A pre-ass eaamen^ will be used to determine appropriate 
if placement of each student in the learning continuum, 

« 

.2 % formative acsessment will be given to measure student 
>^ progress in terms of students' accquiring and retaining 
-/V' stated learnings. In addition, it shall be used 
Vj? to determine schedule of advancement for individual 
students. 
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2. Appropriate records will be kept of criterion-referenced 
data. 



The Assistant Superintendent for Instruction shall be responsible 
for directing the planning and deslgningi implementing > and 
evaluating courses of study and support systems on a systematic 
basis. 

Every effort will be made to coordinate the process with 
the Tempe Elementary and Tempe Union High School Districts. 

Courses of study will be considered and adopted by the Governing 
Board prior tc their implementation in accordance with appropriate 
Arizona statutes. 
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PROGRAM EVALUATION PLAN 

October, 198A 
( revised version ) 
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CURRICULUM AND INSTRUCTIONAL PROGRAM EVALUATION 



Systematic program evaluation serves three purposes:/ (i) to 
determine if the curriculum meets District standarda^", (2) to 
jetermine if student achievement of curriculum objectives meets 
or exceeds District expectations, and (3) to determine if the 
instructional program is effective in meeting identif ied^- ^ s^-^^ 
instructional needs. - ^ 

In conducting program evaluation, two components must be 
considered curriculum and instruction. Curriculum program 
evaluation will focus on the student learnings and objectives 
specified for all subject areas, grades kindergarten through ^u^^^Jjr*^ 
eight. The content of the curriculum is outlined in a scope and p^^^^ 
sequence chart for each subject area. In most areas, 
corresponding assessment tools are available. Instructional 
Program evaluation will focus on the manner in which objectives C 
are met. The instructional program includes such variables as ^c^^^Itf^ 
the amount of instructional time, the instructional materials and^^V^*^ 
resources used, methods of teaching the content or skills, and ^ 
supplemZ%ntal support services and programs. 

Program evaluation efforts will take place when scopes and 
sequence charts are available using the timelines outlined in the 
District's Five Tear Plan for curriculum review. The Five Year 
Plan Identifies by year curriculum areas which are in the phases 
of "planning," "design," and "try-out, program evaluation, design 
revision , implementation . " 

Curriculum Program Evaluation Criteria 

There are four criteria levels to be used in curriculum program 
evaluation. A description of the levels and the evaluation 
criteria follow* 

Level 1 : f '^uurricuxum Completeneaj> 

an Animvi^ basis the euiiri curriculum will be reviewed to 
determine if all ne.eded subject areas are included, and if 
instructional time allocations are appropriate. Subjects will be 
added or deleted, and time allocations will be modified according 
to results of the va*"* -.tion. The evaluation criteria are: 

1. A ct-ise of study has been outlined for all 
curriculum subject areas considered necessary 
for students^ present and future functioning 
in society. 

2. The amount of instructional time allocated to 
each subject area corresponds to priorities of 
the community/Governing Board. 
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Level 2j f Subject/Strand Completeness^ 

Each currrcuxum suojecL Area or strand will be evaluated on a 
cyclical basis according to the District's Five Year Plan for 
Curriculum Review. Modifications will be made if the evaluation 
criteria are not met. 



The evaluation criteria are: 



1. All strands of the subject area have been identified. 

2. Strands have been ^'weighted*' in terms of relative 
importance. 

3. "Weighting" of strands corresponds to students' 
developmental needs and societal expectations. 

Level 3; (^ubject /Strand QuaTT^ 

Content an ii piateuieiifc a f a bj e t iives (scope and sequence) within 
each subyjoct and strand area will be evaluated on a cyclical 
basis according to the same schedule established for Level 2. 

The evaluation criteria are: 

1. Student needs and interests are reflected in 
the objectives. 

2. Competencies needed to function in society are 
included in the objectives when appropriate. 

3. Recent research and knowledge related to the 
content of the subject/strand are reflected in 
the objectives. 

«. Objectives are consistent with District philosophy 
and community values. 

5. The sequences of objectives and assignment to 
.grade levels is developmentally appropriate. 

Aj Student Achievem^t of Subject/Strand Learnings 
E/aluations iilll ue taiiaucted using an established timefrane to 
determine if students at each grade level have acquired the 
knowledge and skills identified in the scope and sequence. In 
the basic skill a^^eas evaluations will be conducted annually. 
In ai: other subject/strand areas, evaluations will take place 
accoroing to the scheduled outlined in the District's Five Year 
Plan for Curriculum Review. Figure 1 depicts the curricular 
program evaluation schedule. 
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The evaluation criteria are: 
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1- ?i urvty . ^ vel measures of student achievemen t 
of curriculum oojectives are appropraltc. 

a. Tests cover an adequate number of 
objectives from a given curriculum 
area. 

b. Test items measure learning outcomes 
described in curriculum objectives. lOf^ 

c. Mastery criteria are appropriate. ^ ^tf^ 

2. A na^^ntiy nf {^|>^^iH^n»« enrolled in the District 
achieve mastery of identified grade level objec- 
tives. 

a. At each grade level at least 75% of 
students in the District iAtef^70-l00X 
o f tested curriculum obje ^ives in a 
given st y'nnff ~ • 

b. At least 75% of students receiving 
instruction below or above grade 
level will master 70-100% of instruc- 
tional objectives derived from asses- 
sing student performance on ^f f-grade 
level curriculum objectives. 

c. Demographic charac teristici of students 
not meeting grade level mastery 
criteria reflect the same demographic 
characteristics as the total school 
population. 

3. A majority of the students in the District meet g>'^^^^^ 
' national achievement standards. \'^^^ 

a. At least one-half of the total number \:> 
of students in the District rank at or 
above the 50th percentile on National 
Percentile Rankings on the Iowa Tests 

of Basic Skills^ 

b. No more than one-quarter of the total 
number of students in the District 
rank at or below the 25th percentile 
on National Percentile Rankings on the 
Iowa Tests of Basic Skills. 
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Instructional Program Evaluation Criteria 



The Instructional program defines the means by which students will 
acquire the knowledge and skills specified In the curriculum. 
The two levels of Instructional program evaluation are quality 
and effectiveness. 

LEVEL 1: Instructional Program Quality 

In evaluating the quality of the Instructional program the major 
question being addressed Is whether or not personnel at the 
District and school levels are providing an adequate instructional 
program. 

1. District level evaluation criteria: * i» • 

a. Course of study guides are provided for 
each curriculum area that Include grade 
level performance objectlvest and recom* 
mended instructional time allocations. 

b. Enough staff ar$ provided and other needed 
support staff members for each school; 

c. Adequate resources are provided for Instru- 
tlonal materials. 

d. Instructional support services are provided. 

e. Staff training needs are assessed and neces- 
sary training provided. 

2. School/classroom evaluation criteria: 

a. Teachers are teaching to the objectives ^^-^^ 
specified in the curriculum. ^^i^^O 

b. Teachers are following recommendations 
for instructional time allocation. 

c. Instructional materials and resources are 
available and are used appropriately accor- 
ding to learning outcomes specified in 
objectives. 

d. Teachers Msess student performance related ij2>>^^ 
to specified objectives and use evaluative 

data to plan instruction. 



e. Teachers use principles of learning in 
delivery of instruction. 
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f. Student performance is routinely monitored, 
and records are kept. 

g. Remediation is* provided when needed. 

?r "51 BEST copy AVAILABLE 



Instructional interventions are evaluated to 
determine if student achievement is influenced, 

1, A plan for use of support services is developed 
and is operational • 

Level 2: Instructional Program Ef f ectivenss 

The primary measure of effectiveness is student achievement. If 
the District student achievement standards are being met, (refer 
to Curriculum Program Evaluation, Level 4) then the instructional 
program is Judged to be effective. If standards and 
expectations a;'e not being net at both the District and school 
levels, intervention should be planned which corresponds to the 
outcome of the instructional program evaluation, Level 1, 



Curriculum and Instructional Program Evaluation Procedures 

The Assistant Superintendent for Instruction will be responsible 
for supervising the evaluation of the curriculum and 
Instructional program. 

Curriculum 

The *'Five Tear Plan for Curriculum Review^ outlines a schedule 
for planning, developing, implementing, and evaluatling 
Individual curriculum areas in the program. In the "'Planning" 
phase of the cycle, the scope and sequence of a specified 
curriculum strand will be reviewed and evaluated according to the 
criteria outlined for Levels 2 and 3 of Curriculum Evaluation of 
this policy. 

In the ""Design^ year phase curriculum revisions will be made 
according to the recommendations resultling from the above 
evaluation, A draft version of the revised scope and sequence 
will be submitted to the Board for interim adoption if needed. 
The fiMl activiy in the "Design" phase will be the development 
or refinement if needed of an evaluation tool to be used to 
measure student achievement of a sample of curriculum objectives, 
Th:* criteria outlined in Level 4, Curriculum Evaluation, of this 
policy should be applied in the developpient of this evaluation 
tool. 

In the third year of the cycle a "try-out" of the fcope and 
sequence and evaluation tool will be conducted if major 
refinements have taken place in those cases where there are 
major changes and when possible, pilot schools and/or classes 
will be Identified for the "try-out" phase of development. 
Student performance data and evaluative feedback from teachers at 
pilot sites will forjj the basis for the final review/revision of 
the pilot scope and sequence. 




At the end of the third year the final version of the scope and 
sequence will be presented for Board adoption for the entire 
District. 



During the "Program Implementation" phase of the "Five Year Plan 
for Curriculum Review" student achievement data will be gathered 
and analyzed according to the criteria specified In Level A, 
Curriculum Evaluation, of this policy. 

Instructional Program 

Student achievement data will be analyzed according to the 
timeframe outlined in Figure 1. The Instructional program will be 
evaluated according to criteria outlined In Levels 1 and 2, 
Instructional Program Evaluation, of this policy. 
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FIGUBE - Program Assessment Timeframe 



Subject Area 



Annual Annual Annual 
Pre/Post 

Sep. -May Jan. May 

K-e 1,3,5,7 1-8 



Bl-Annual 
Even Yrs 

March 
K-8 



Bl-Annual 81- 
Odd Yrs Annual 
Even years 

March 

K-8 7,8 



Hath 

CASS, Study/Ref 
CASS, Written Comm. 
CASS, Grammar 
CASS, Spelling 
CASS, Handwriting 
CASS, Reading 
CASS, Literature 
CASS, Vocabulary 
CASS, Oral Comm. 
( Listening/ 
Speaking) 
Computer Science 
Science 

Social Studies 
Career Education 
Home Economics 
Industrial Arts 
Foreign Language 
Health Ed. 
Music 
Art 
PE 



All 
All 

All 



All 

All 
All 



All 



All 



Ran 



All 

All 



Ran 
Ran 



Ran 
Ran 
Ran 



Ran 
Ran 

Ran 



Ran 



NOTES: ALL : All students will be tested. 

RAN 's A random sampling of students will be tested, 
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OPTION 1 



COST 

Purchase of Adams County System (CMI 3000) $ 5,000,00 

^ Purchase maintenance package from Adams County 

(yearly requirement) 2,50v.,00 

Purchase maintenance support and Interface to 

current HP ?ystem 5,000,00 
Purchase additional hardware In order to Install 

CMI 3000 system 50,000,00 

Sub-Total $ 62,500.00 

»» By selecting Option I, this allows the district to purchase and Install 
the CMI 3000 package along with the necessary hardware. No scoring or 
reporting would occur with this option. 



56 

ERIC 



OPTION II 



COST 



- All components of Option I 

AND 

- Assign coordination of CM! 3000 System 

- Hire new or utilize existing clerical staff to 
Input district CASS and MSS objective? 
(estimate 200 hours at $8.00 per hour) 

- Purchase necessary paper products (I.e., answer 
sheetfty etc.) for test scoring and reporting 

- Hire personnel to score answer sheets and 
generate reports (estimate 50 hours at $10;00 
an hour) 

- Staff orientation and training 

- Miscellaneous Implementation costs (i.e., quality 
control preparation of test directions, consultant 
services) 

TOTAL 



Sub-Total $ 62,500.00 



$ 0 - ($40,000.00) 

^ ,600.00 
7,500.00 

500.00 
5,000.00 

5,000.00 
$ 82,100.00/$122,100.0( 



By selecting Option II, the district would have the CMI 3000 Installed and 
operational. The CASS and MSS cests would be scored and minimal reports 
generated. Staff orientation to the new system would be provided on a 
limited basis. Quality control would need to be done at the school level. 
This choice provides the option of either assigning the coordination or of 
hiring new staff to coordinate. No teacher generated tests or custom 
designed reports would be possible. 
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OPTION III 



$ 82,100.00 
AO, 000. 00 



COST 

- All components of Option I ^ 62,5CD.OO 

- All components of Option II 19. 600. 00 

M2 Sub-Total 

- Hire full-time testing coordinator I 

- Data Pro.essing/Quality Control personnel 20,003.00 

- Hire full-time secretary to Coordinator 15,000.00 

- District Consultant to analyze reports and 

• determine needed revisions and custoir reports 2,000.00 

- Programming services to customize reports and 
tests for CM/. 3000 program changes 10,000.00 

- Personnel or consultant for statistical analysis 
of results and for all program evaluations 5,000.00 

- Additional hardware to provide for teacher 
generated tests at each school site 

- Additional staff developing and training 



25,000.00 
5,000.00 
TOTAL $ 20^,100.00 



S>r.;rs:i./^%-;rj:;:%;:tr:.:::i:r:;/j;.s;;y;/-:us^ 



58 



APPENDIX H 



I 



iQREME acaooL msiRicr 

MSCBAMDIK 



TOs iMtructional Coaputer Coanittae 

TOO,. Mike rsUy ^//^ 

01^. ^26-85 

^^^^^ MMIGEMEOT S!^ 

^ U«tad on the following pages to eee if 

Please t«vi«i the task. «>d«c^tiM Ustea^ The tasks are 

SS^adBmately ooi«r your P^ceptions or zxzv^ j«u. 
S^itS^ the following headings: 

X itanaacBent structure 

sSSTaMulsition and Baintan^ 

S: ?2rLei^ 

0. scoring, reporting »dja^ 
E. Staff dHwelopBant training 
P. Progt» ewalustion 

Aa vou -ay not. , thar. is toaa f or *t«»ini«gdat«j, J^SJ^^ifd^- 

SiS'i'^tlSS^Ji. I wiU be r«dy to -.are 

norning, Ortober 2, 1985 •^J^^"^ activities to date, 
with you the prellaiinary report on our 
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Page 2 



A. M^Maflwmyr strdctore 



Date T»» ^ . 

SS^'is SSTrf pe«««l c*ain of oa^ 



Itent TBooen 



Oct. - 
Nov. 85 



Oct. - 



assist ulth aoquUiticn 
lasntify p.r««nel to ca^^ob^c^ mrl«e P«.y 



M-nti£v t»r«3nn«l to perfom *ta entry 
ref er««d to te«t ItiM 



itov. - 
Dec. 85 



resalts to echooU and t«che« 



Nov. - 



Nov. - 



Identify end ewign P«f«»»l^J?,SSffive 
®^ 5S>ogr» evlimtion jOan 

instalUtion and '^iJi^S^tii^^ 



85 



Budget 
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page 3 

Objective; Tb purchMe «>d in.taU the OI 3000 .y.t-n for «»ring t«ts 

and reporting results g^ ynslble 

Party 

Pate 

Contact M«s County, Colorado Sd»ol Dlrtrict 
#12 to obtain procaduras for purchasing systan 
and naintenance suciport. Budget * 



Oct. 85 



Oct. 85 



Oct. - 
Nov. 85 



contact P«taine, Inc. of California to obtain 
IJSoaduras for purchasing naintenance support 
for student inf onnation base 



?K!'.SSS, »5e., additional umovy tmOB. 

etc. ) Budget « 70 ^000 or more 



HO. - Prepare contract paper, to obtain oa 3000 
DK*. 85 ObSin Goiieming Board approval 

Budget - 7,500 



xtrmr^rm ««tract oapers to obtain Pertaine 

Obtain Gogering Board approval . 5^000. 

- P.eoare data processing staff training 
85 JlSSe for E^ing «I 3000 on line 

Budget - 

Dec. 85 Install OfL 3000 systan 



Budget - - 

Dec. 85 UaJc student infomation base software 

Budget - . 

Jan. . Identify reports to be printed and 

Peb. 86 distrihited « »0- 
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ite« HQn8ibIe 
Party 

Date ^ 

^. 86 Prapare «««r tey. for aach t«t to be «»red 

Budget 



Mardi 86 



Prepare pilot wrvey test fn«der to 
SSS Scanning* acoring and reporting field 
test 

Budget - 

.urA 86 conduct field test of ^OOOacoring. 

B^letf for accuracy and oonpletaness 

Budget • 

^ 86 PreP«« 

SS^andpcoject peracnnal and ha r>>m 

«SS S !iarri987-90. mcluflas »ry 
SiSaiS. «aiStion of additional HP 150 
SSSS f 5^-cher uae, taff training and 
progran ewaluatioo needs. 

Budget 



CniblM)c4/D45 
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Page 5 



curriculvw ar«a nagpensible 
SS . 86 ge!«ration in language arts and mth 



Budget - 



Or, 85 cross »fer«ioe to scope and sequence in 
Sn. 8l art. «d«2 for each grade 

Include any statistics. 

Budget • 

March 86 D-velop skiU '^^J^ ^ 
language arts and oatn 

Budget < 

M-reh EoBiine additional it« hanks in other 
oSi«l« areas for pcj«ibl; punchy 
Determine iwropriate siie of the itaa 
collBCtion for esch area. 

Budget 



Pxware plan for itan hank dwwlepnent/ 
n^iaae in other curriculon areas 
folloiiing five yaar curriculm plan 

Am 86 Prepare plan for possible entering of 
Avq. 86 ^P^iJ^Hn OO 3000 test hank 



Budget 



Budget - 



Aug. 86 



Determine test develognent and it«\ 
bank mnaganant •yst^i including hew 
SSt itSrSl »i qpdat«l and 

update it«ns 

Budget - 

jua. 86 Identify review procedures for ediUng, 

^ aiding or deleting it«s . 
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Date 
Aug. 86 



iteaoonsible 

Task 

EBtabliah iecurity procedures for •coessibility 
to item bank 

Budget - 



S^iater curriculum areas such as gifted, etc. 



Budget - 



Caif^ruk6/D45 
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Oct. 85 - 
Jan. 86 



Party 

Pate ' 

^Bth and language arta?) 

Budget - 

Buflget - . 

. . - _ Atractar oodea for lub-curriculwi 
Kyrene 

Budget - _ 

SSt^Sr jSrTOa 3O0O *t. 

'oaae 

Bi^t - ^ 

SSSSty J«^. 

Budjet - — 

Budget « 



Jan. 86 



Jan. 86 



Jan. 86 



Oct. 85 - 



Canc)'Baak7/D45 
c/W850926/*B 
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itegoonsible 
Party 

Pate ^aai^ 



Jan. - 
Feb. 86 



Jan. - 
Feb. 86 



«0M r^erencad »v test it«» into OO 3000 
data base 

BoOget » . 

Prepare student data base ^^^^^ ^^^0 
iSJment of student ID tninbers. ftiter into 

cyi 3000 systsm. 

Buc^t - 



j.„ . prepare teacher ID system and enter into 
Feb. 86 00 3000. 

Budget 
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Objective: Tb orient staff and teachers on new student testing syston and 
new report fomats. 

Responsible 

Pate Task Party 

Mov. - Fr^are staff dsvelapnent plan and associated 
nec. 85 activities regarding new systan and retort 
fomats 

Budget - 



Nov. - Identify appropriate dates to oondust staff 
Dec. 85 inaervioe 

Budget - 

March 86 Identify and prqare copies of new r^rt 

fomats to use in staff orientation including 
objectives list, class nastery profiles, 
student profiles and district smnaries 

Budget - 

Hard) 86 Develop test directions for teadier to follow 
in preparing answer sheets 



Budget - 



April 86 Establish staff training for using system 
to develop taadn&r made tests 



Budget 



Caqp/ras)c9/D45 
cAD)c/850927/an 
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Ob''.<a tlvet To establish e oon^iuterlzed nanagcrattit systan for conducting 
student, t«acherr and school ujqparinns of test perfonnancft 
in relation tc qiecific program goals 

Responsible 

Pate Taak partv 

Vov. - Contact Fertidne, Inc. to detitxnina if softHBre 
Dec. 6d is svailab]^ to conduct staiistical anaylsis. 

Prepare contract to dte^tPlap qiecialised/cus- 

tanixed softwere if nona mrailable. 

Budget - 



Jan. - IdKitify ^pacific prognn evaluation infocm- 
March 86 tion desired foUowing October 1984 Fiogrvi 

Bvaluation Plan and peepere ifidated tine table. 

(Curriculim and instructiooal evaluation) 

Budget - 

2nd year Establish plan for conducting betMeen school 
or beyond ogqperisons of echievenent results, end 

betueen teadiers (within grade level) ccnijari- 

sons of achievement results 



Budget ■ 



2nd year Develop adiievenent prediction' aiodel if 
or beyond ^jpropriate software is available and 
ccnpatible %iith OII 3000 systan 



Budget - 



1986-S7 Pilot test between sdiool ooBparisons and 
or later betjem teadier (within grade level) can- 

parisons of adiievanent results using 

m>ropriete c'-atistics 



Budget 



1987 or Pilot test adiievenent prediction nodel 
later ^rcwidnd testing and evalvntion unit has 

appropriate personrel to conduct su^ 

a test 



Budget 
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McMinn.ille School District 



District Demographics 

McMinrville is a community of about 15,000 in Northwest Oregon. The city 
is about 90 miles from Portland and 60 Riles fr<Mi Salem, the state capitol. 
The district serves about 4,000 students in six elementary, one junior high 
school and one high school. The six elementary schools serve from 200 to 400 
students, the Junior high school serves about 900 and the high school serves 
about 800. The school population has been fairly stable over the last few 
years. 



Description of Current Testing Scheme 

Purpose. The primary purpose of the criterion referenced testing program 
is to provide ongoing skill level formative information to teachers in order 
to assist in planning anC instuction in grades K-6. K secondary purpose for 
the CRT program is to provide summative information at the end of the school • 
year on how students are performing on skills which have been locally 
identified as essential for 

Source of Items. The CRT program is based on the Harcourt, Brace, 
Janovich reading and Heath mathematics textbook series. Pubxishers tests were 
provided with the text series. These tests w^re locally rewritten as needed 
and placed into multiple choice format to facilitate computer scoring and 
stuciert record keeping. The number of test items for each goal vary according 
to the number of performace indicators estah ^ shed for each goal. A numerical 
break down of course goals, performance indicator skillL and test items are 
containeo in Appendix A. The test items and textbook activities and 
supplementry matet>ls are cross-referenced to a locally developed curriculum 
for both rear j and mathematics. 
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Testing Expectations . Every classroom teacher (K-6) is required to 
administer pre- and post-tosts (fall and spring) in reading and mathenatics. 
There is a single torn for fall and spring testing which covers all 
curriculum concepts for the entire school year. Periodic tests 
(approximately every nine weeks) in reading are optional at the primary level 
(K-3) and required at tlie intermediate level (4-6). All teachers in grades 
K-6 are required to administer a mid-year math test. The periodic or mid-year 
tests include curriculum concepts covered up until that point in tii« school 
year. CRT tests are administered according to preset timelines. These 
timelines, called curriculum mapping guidelines (see Appendix B) , are set each 
year by a committee of teachers representing each grade level. This committee 
considers factors such as the school calender, past experience with teaching 
the reading and math curriculum, and past performance on the curriculum as 
measured by the periodic tests to set the timeline. 

Testing . The district pr.int shop prints all tests req*iired by the 
curriculum mapping timeline. These tests are delivered and stored in each 
builcUng prior to the beginning of each school year. The district also 
pre-slugs answer sheets with student information. Bach school is reponsible 
tor submitting class lists, when lists are received, the ScanTron answer 
sheet is preslugged and given a class designation for use the rest of the 
school year. Teachers are responsible for obtaining and inventorying all test 
inaterials (answer sheets and tests) prior to the test administration aate. 
Principles are responsible for distributing test materials to teachers. 
Teachers are responsible for testing, collecting and maintaining quality for 
their classroom's ScanTron test forms. Completed tests, including makeups, 
are turned into the principal to be forwarded to the district office for 
computer scoring within one week of the testing date. Makeup tests are not 
computer scored after this point. 
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Training, A one hour district inservice on test administration, ScanTron 
marking, test scoring, interpretation of results and general procedural 
concerns is given at a K-6 general meeting. A testing manual explaining the 
procedures of the program is available to teachers. In addition, the computer 
testing specialist is available to buildings for individual inservices or to 
answer individual concerns. 

Scoring a nd Reporting . Upon receiving tests, a student aide at the 
central office checks for proper formatting, erases stray pencil marks, and 
darkens light marking of answer forms. This takes approximately 20 hours for 
the entire district ScartTron forms for each of the Math and Reading tests. 
Turnaround time for scoring the tests from a class can be as short as one day 
or as long as 2-4 weeks depending upon computer down time and quailty of the 
ScanTron forms. 

Three copies of the results are produced. One copy of the results is 
provided for the teacher's use in formative evaluation of goal bared 
instruction, one copy is provided for the principal, and the third copy is 
used by the district office for curriculum evaluation. Teachers receive 
printouts showing course goals master -d by each student (Appendix C) . Thesc^ 
lists provide both alphabetical order and rank order scores. Test results are 
sh&red with students and parents in individual conferences and must be 
considered in grading, promotion and retention as mandated by district 
policy. 

Student testing information is kept on disk. Other reports generated by 
this system include longitudinal profiles of individual student grade and 
district skill acquisition. Although not used at present, the software has 
the capability of generating point biserials and Rasch calibrations for each 
tost item. 
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Software and Hardware , The list below summarizes district software and 
hardware features and costs. 
Software Features ; 
o Preprints answer sheets 

o Scores answer sheets by high speed scanr r 

o Reports test results using district goals and competency criteria by 
class, grade and individual student 

o Monitors items for level and quali^ 

o Manages test information from year to year 

o Menu driven 

o Automatically updates test statistics as new information is processed 

o Provides support for reporting Rasch scaled scores 

o Uses a data base structure to link test information to the students' 
current class and grade. 

Hardware ; 

o Apple He Professional, IBM PC or TRS-80 3/4 
o High Speed Scanner 
o Printer (150 cps) 

Cost ; 

o Software System $1200.00 
o Annual Maintenance $240.00 
o ScanTion Answer Sheets $117.00 
o High Speed Scanner $2000.00 
o Printer 1000.00 

o Self Feed Unit $700.00 

Standardized, Norm-Referenced Testing . The District also admisisters the 
SRA test series to all students in the spring of each year. Whii« the CRT 
provides formative and summative evaluation information which is directly 
related to the curriculum, the standardized test provides summative 
information on curtent student/district standing. The CRT's have been 
c'^oss- referenced to the SRA to determine if similar goals are being 
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addressed. It was found that the CRT's aeasure a broader spectrum than those 
tested with the SRA. In some cases, however, skills tested on the SRA were 
not found to be covered in particular grade level course goals. Teacnsrs have 
been advised where the goal-based curriculun does not address a SRA goal and 
are expected to supploaent the cirruculum to cover this skill. 

History Of Implementing The Testing System 

Impetuse s for the Testing Scheme . During the school year 1980-81 several 
developments occurred which influenced the testing direction taken by 
McMinnville. First, the district h:.d been concerned about a decline in 
reading comprehension scores on standardized tests. Second, 1980-81 was the 
statewide textbook adoption year. Third, McMinnville School District became 
interested in the direction the Valley Education Consortium (VEC)* was taking 
to develop goal-based curriculum guides. It was felt that both the textbook 
adoption and the goal-based curriculum could serve to improve perf raance on 
the basic skills. 

The Superintendent of the McMinnville School District, after participating 
in VEC meetings, also became interested in the use of computers for measuring 
student acquisition of skills and providing feedback to teachers on student 
performance and progress. As a result of this irterest, the McMinnville 



* The Valley Education Consortium is an association of institutions in the 
Mid-Willamette Valley which set about the task of developing instructional 
programs in the basic skill areas of reading, writing and mathematics. 
Seven school districts in Marion, Polk and Yamhill counties, the Education 
Service Districts serving these counties, western Oregon State College, and 
the Teaching Research Division of the Oregon State System of Higher 
Education participated in the development o' guides that were intended to 
assist, teachers in planning instruction and monitoring student progress. 
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Superintendent approached VEC for seed money to develop and implement a 
computer testing program. With the seed money, the district purchased a 
number of computers and contracted with an independent software developer to 
develop the system. 

This plan called for developing detailed scope and sequences based on VEC 
skill continuums and then systematically assessing student progress on these 
skills. Although tests would be in fixed forms, scoring, reporting and 
tracking student progress would be by computer. But since these plans were 
developing at the same time as the new textbooks were being selected, the 
curriculum and tests had to be developed/revised after the adoption and then 
cross-referenced to the next textbook series. 

Reading Adoption (School Year 1980-81) . The Reading Facilator, (an 
elementary principal, expert in reading and assigned for this purpose) with 
the assistance of the other elementary and secondary principals, appointed a 
Reading Coomittee consisting of two teachers in each of grades K-8, a high 
school reading specialist and a special education teacher from the Learning 
Resource Classroom program;. 

All of the teachers assigned to this committee were recognized by their 
principals and colleagues to be outstanding reading teachers or ones whose 
particular teaching assignments were primarily reading instr xion. This 
committee was to select textbooks dur'ng school year 1S80-81 and to develop 
curriculum guides and tests during school year 1981-82. The Reading Committee 
met on a weekly basis, usually on Wednesdays after school from 3:00 to 
4:00 p.m. 
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After presentations by textbook publ'.shers and evaluations by the entire 
staff, the connittee aade the final choice to adopt the HBJ Program since they 
felt it to be stronger in teaching the reading comprehension skills felt to 
need anphasis in McMinnville. 

The recommendation made to the administrators by the Reading Committee 
stated that ^ile HBJ Reading Program was being recommended for use 
district-wide in Grades to develop a uniform developmental reading 

program. An added benefit of a uniform program would be that of providing the 
same program to students who transferred from one school to another within the 
district. Prior to this, many principles had allowed teachers to use the 
reading program of their choice or to pilot new programs instead of using the 
adopted program* 

The only modification in ^iiis place was that the kindergarden teachers 
petitioned to use the Lippincott Series instead of the HBJ Program since they 
felt it to be strongi.r in phonics. This request was appproved, but within the 
understanding that the kindergarden teachers would be required to adhere to 
the District Testing Program once it was in place. 

Developing The Curriculum Guide (School Year 1981-82) . The reading 
committee began working on the Reading Curriculum Guide, using the VEC Reading 
Goals and objectives as the core worf: for the document. The following 
procedure was used. 

o Reading Committee members met with all district teachers at their 

particular grade levels to review the VEC goals and objectives, 
o Based on the teachers* knowledge of curriculum and actual practices 
in the classroom, these goals and objectives were revised and 
additions or deletions made where needed* 
o A consensus was reached by the teachers at each grade level on goals 
and objectives. 
O 70 1--^ 
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o Following general acceptance of the final goals and objectives, 

committee MOibers were given release tiae (about 2 days each} to 

cross-reference the HBJ series to the curriculum, 
o After the ccamittee senbers had keyed the skills to the objectives, 

columns were left on each page of the guide for cross-referencing 

test items to the objectives at a futu«re date, 
o The Reading Guide was the first of a series of curriculum guides 

developed in the district, and it became the prototype for future 

documents. 

o The final curriculum guxde included all of the following information: 
o Program and Course Goals 
o Scope and Sequence Summary Table 

o Grade Level Guides which included Objectives, Textbook 
References, HBJ Unit References, and and Test Itrs 
Cross-references, 
o Reading Curriculum Mapping 
o SRA Test Cross-reference 
o Criterion-referenced Test Copies 
Criterion-Referenced Test Development (School Year 1982-83) , At the end 
of the 1981-82 school year, a new superintendent and deputy superintendent 
were contracted by the School Board to begin working in the district during 
that summer. The new superintendent came from another district where the 
orginal thrust for goal-based curriculum had developed; and, as a former 
member of the VEC Board of Directors, he was familiar with current aistrict 
activities in curriculum writing and test development. As a member of the VEC 
Board, he had been inrtrumental in granting the request for seed money to 
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develop the c<»iiputer testing system in McMinnville. With the bwking o£ both 
the superintendent and the deputy superintendent, who had a strong background 
in curriculun and instruction, the conBDittee*8 efforts took on new energy and 
momentum. 

The next step by the Reading Cosnittee during the fall of the 1982-83 
school year was that of developing the criterion-referenced tescs for 
assessing student progress and program i]q>lementation. Because the new 
reading adoption included a series of test booklets, the commi*''^^e decided to 
use these "author tests" as the basis ol the testing program, rather than 
reinventing ihe wheel. The cosmiittee obtained permission from the HBJ 
publishers to rewrite and revise the author's tests. These tests were 
rewritten in multiple-choice format to facilitate machine — )ring. 

Two teachers at each grade level were given release time of approximately 
two days to revise the test items on their grade-level tests and put them into 
multiple-choice format. In consultation with .^i<^ c^i^ter coordinator and 
software developer, the committee determined the number of items to be used on 
each test and the number of choices for answers that could be used. The 
revised versions of the tests were submitted to the teachers at a grade-level 
meeting after school for further sugc ^sted changes and for a consensus 
agreement. The entire test-writing process took about three months. 

After the tests had been rewritten, the district contracted with ihe 
Northwest Regional Educational Laboratory for a "testing specialist" to review 
the tests and advise the committee about the validity of the test items as 
they were written. Subsequmt to this review, additional changes were nade in 
the test items and the tests were retyped at the district office in 
preparation for a pilot test. 
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Foxlowing the final test construction session, the Reading Committee 
members met once age^in in the fall, on a release time day, to cross-reference 
the test items to the Reading Objectives in the curriculum guide, and the last 
column on the guide forms ms filled in. By using release time for this 
curriculum and test writing project, the only cost to the district was in 
substitute pay. 

Pi lot Testing . Tiiro schools in the district volunteered to pilot the tests 
during th^ second half of the 1982-83 ochool year. One school was :sed 
because standardized test results were consistently lower than other 
elementary schools in the district. The other school was used because the 
principal served as the District Reading PaciliUtor and the staff was 
highly- supportive of the project. 

The test had been printed, collated and stapled by the district printer at 
minimal cost to the district, and ScaitTron sheets were developed by the 
computer coordinator and software consulUnt. A ScanTron reader was purchased 
by the district and placed at the district office. 

The periodic tests were piloted at the end of each reading book in the 
primary grades and at ths end of each quarter for the intermediate grades. 
The cumulative pre-post tests were administered to all students in early May 
so teachers would ^«ve the results before the wd of the school year. Student 
responses were recorded on the ScanTron forms by the teachers or other adults 
in Grades K-3 and by the students themselves in Grades 4-6. Completed 
ScanTron forms were sent to the cc^uter coordinator at the district office 
for computer scoring. 
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The conputer coordinator and computer software consultant had developed 
the necessary software to provide the following information to teachers on 
computer printouts: 

o Student performace on each skill area tested. 

o Student results by alpnabetical and rank order, 

o An item analysis for each class, with both correct and incorrect 
responses tabulated for the teacher's use. 

Printouts were distributed to the principals, the classroom teachers, and 
the central office administrators showing both group and individual results. 
Teachers received printouts shown in Appendix C. 

At the end of the year members of the Reading Committee met again on 
release time to review the tests. Revisions to test items were made based on 
teacher recommendations ana information fr^ the item analysis. 

Major Issues k Concerns About The Tests . At the end of the pilot year 
there were teachers in both schools who were positive and some who were 
negative about the testing program. The positive teachers indicated that the 
information obtained from the test data were helpful in planning for 
instruction and remediation. By setting high expections for their students, 
they found that the students were performing at a higher level. 

The teachers who were negative about the program indicated that the tests 
were too time-consuming, that their creativity in teaching was being destroyed 
because of the structure being imposed, and that they already knew how their 
students were performing without the formality of a written test. 

The program was perceived to be more successful in one pilot school than 
the other because of the administrator's interest and investment of time in 
the project and the willingness of the teachers to cooperate in one of the 
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pilot •chools. Many of the teachers at the other pilot school chose not to 
complete the pilot program and their administrator allowed teachers to 
discontinue ^he program if they wished. 

An analysis of the test data from the successful pilot site, and a 
conqparison of the standardized test result during the pilot year to the 
previous year's results, indicated that the program had been extremely 
successful for those teachers who completed the project. Even the teachers 
who were negative about the testing program showed gains on the standardized 
tests after using the testing program as outlined. 

Implementing the New Program (School Year 1983-84) . In the Pall of 1983, 
vrtien the "National at Risk" report bec;ime public and school districts across 
the nation began searching for ways to improve public education, the 
administration in NcMinnville felt it had a head start on most districts with 
its goal-besed curriculum and criterion-referenced testing programs in 
Reading. After reading much of the research on effective schools, che 
administrators in the district agreed to implement the testing program 
district- wide. During the 198>84 school year, elementary teachers were given 
inservice training on the effective school model for our district anu asked to 
do the folowing: 

o Use whole-group instruction for introducing grade-level objectives in 
Reading. 

o Us» skill grouping (rathc^r th-^n ability groupings) to teach the 

.irade-level reading skills, 
o ftdminicter the District Testing Program in Reading to assess student 
Itiisition of the skills. 
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o Use the computer printouts o£ test results for the following purposes: 

• Diagnosis of student needs. 

. Identification of skill grouping for instruction • 

• Evaluation of student progress. 

• Evaluation of their reading program. 

o Follow the Curriculum Napping that provided a guideline for pacing 
instruction and for a ninistrating the test throughout the year 
according to a timeline. 

o Dse a mastery approach with a goal ot 80 percent criterion level for 
at least 80 percent of the students. (This goal was arbitrarily 
selected based on the effective schcK>l research administrators had 
read.) 

After the program was implemented district-wide in 1983-84, administrators 
3 till tended to "give in" to teacher resistance and allow some teachers to 
avoid giving the tests. Ultimately, an administrative directive came from the 
superintendent for aJl teachers and principals to comply with the program. 
This resulted in district-wide compliance, but it also made many teachers 
angry that the program was being imposed without the fri^edom to choose whether 
or not to participate. District-wide compliance could have evolved through a 
natural course of events over a period of years as the program proved itself 
to be effective, but the administrative directive certainly expedited the 
process and facilitated the implementation of the program. 

The diagram in Appendix E illustrates the whole-group instructional 
approach teachers were asked to use in teaching reading. Teachers were 
expected to pretest all students, identify the ones who demonstrated mastery 
of the skills on the test and send them to enrichment classes, then teach the 
remaining group of students in a whole-group approach. After teaching a 
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concept, teachers are expected to test, then reteach the students who did not 
master the concepts. After the teacher has completed the teach-test-reteach- 
retest cycle two ot three tines, students who still have not nastered the 
skill are referred to the learning resource teacher for renediation. 

Math Program. During the time the Reading Program «#as being implemented 
in 198?-84, a new conmitteee of teachers was appointed to follow a similar 
procedure in adopting a New Math Program and ''••^eloying t curriculum gui'^'* and 
testing system f acccnpany it. 

Under the liirc.txon of the principal who served as the District Math 
Facilitator and a lead-teacher who was recognized as an outstanding Ma«-h 
teacher, the committee adopted the Heath Math Prcqru.. The VEC format was 
aoain used to develop the Math Curriculum Guide and to correlate the math 
skillr to the math objectives. 

The development of che Mfth Testing Progra was "insider ably l^ss 
difficult than the Reading Testing Program. The Heath M. th adoption came with 
a built-in assessment program which was already in multiple-choice format. It 
simply required that the information be programmed into the computer ar 
cross-referenced to the curriculum objectives to provide the printouts of 
student results according to skill acquisition, as In reading. 

Th:j program was in place and all teachers were required to edh .e to the 
mapping, guidelines to£ testing during the following school year (1984-85). 

Effects o f the CRT Te/^tnq Programs . The district has noted several 
effects o cheir curriculum alignment effort. 

1. Standardiz ed Test Results . The district reports that as a direct 
result o* the district's efforts in curriculum alignment, the 
tttu^^ents at the ciementary levels have, in most cases, shown ar 
n crease in both Reading and Math scores (See Appendix F tor an 
wjuunple of such results.) 
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2- Cr iter ion-Re ferenct^d Test Results . Teachers are achieving the goal 
of a criterion Ir^vel of 80 percent for at least 80 percent of the 
students on the Periodic Tests. At the intermediate level (Grades 4, 
5, and 6), students are not achieving the 80 percent level cn all the 
skills on the post-test. Teachers attribute this to the variability 
in retention anong the students and to the larger number of concepts 
being taught and tested at these levels. 

3. Teacher Attitudes . During the first year of the aistrict-wide 
implementation of the Reading Program, teachers were extremely 
negative about the program. Concerns expressed by the teachers 
centered around the foLowing topics: 

o Grouping ; Hany teachers had not "bought in" to the whole-group 
instructional approach. The previous reading program 
(Hacmillan) had an individualised approach. Although the BBJ 
Reading Program is designed for a whole-group approach, few 
^ teachers had attempted to use it in that manner before. Most 
had adapted that reading program to their ability grouping 
techniques and found the prosp^t of changing teaching 
strategies a little threatening. After a year or two of 
teaching with the %#hole-group approach using shoit-term skill 
groupings, the district reports that the majority of the 
teachers were pleased with their success and the accompanying 
increases in test scores. 

o Curriculum Mapping : Many teachers felt "pushed" to stay ^ 
txack with the mapping requirements* Coomients were made that 
teachers were no longer able to capitalize rm t'«e "teachable 
moment", that teaching was no longer any fun si ice they couldn't 
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take time to teach what they wanted, and that the testing 
process was too regimented. Most teachere indicated that they 
had rarely taught the full year's curriculum prior to the use of 
curriculum mapping. 
® Test Auministration s Many teachers still resist the regimen of 
following a testing schedule, and conplain about the aiaount of 
time required to administer the tests. Some feel threatened by 
the ccnputer printouts being used by administrators to evaluate 
student pi.w. ess and for program evaluation. Th^re are still 
many criticisms by teachers regarding the validity of individual 
test items and the technical aspects of scoring te»ts and 
delivering tt.e printout in a timely manner. 
The district feels very strongly, however,, that the progr^ does 
worK, despite its flaws, and that they have seen the payoffs in the following, 
o Increase in student CRT results, 
o Increase in standardized teat results, 
o Increase in student morale, confidence, and self-ei ^eem. 
o Development and implementation of a coordinated district-wide 

enrichment program, 
o More effective use of the Learning Resoux ^e Centers, 
o More effective means of communicating student achievement to parents, 
o More effective teaching techniques, especically in using a mastery 
approach. 

o reaching more curriculum concepts and skills by following the 
curriculum mapping. 
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Current Status 

General Considerations . The district feels that a program such as this is 
never "done." There are always new needs, new technologies, and new demands 
being made which cause the program to rer^in in a constantly evolutionary 
status. At this point in time the district identifies a need for a more 
efficient and faster scanning device. Increased speed brings about greater 
savings in labor costs and meets the need for more rapid turnaround of test 
lesults bade to the teacher. 

Since the program is curriculum imbedded it must be responsive to changes 
in the curriculum. Although the district has adopted goals/objectives in 
reading and mathematics, the fact r mains that the textbooks used by the 
teachers have tremendous impact on the curriculum and the testing progreun must 
be sensitive to this fact of life. However, that does not mean that every 
time a new texw is identified that the totality of the testing program needs 
to be revised. Rather, as more comprehensive skill and itn banks are 
developed, the district anticipates the ability to develop reliable and valid 
tests by cross-referencing with the new materials. 

The district also fsees impact on the curriculum and tests from the State 
of Oregon's new program which will identify certain "essential learning 
skills" for each student. The intent of the state is to require each school 
to become accountable for the student's learning of these skills. These 
skills will be imbedded in the (curriculum. The district wants to mvke sure 
that the testing program is sensitive to measuring student progress on the 
state identified skiJls as well as locally identified skills. 




Elementar y Grades . At the elementary level the district has identified a 
need for more test items to be developed sc that additional test forms can be 
constructed. Additional forms keep the testing program f;:e8h in the sense 
that tlie students do not beccne use to the older test forms. Further, 
teachers are developing teach-reteach strategies which require more frequent 
testing. 

Recently, the district made plans to move towaud a 6-8 grade middle school 
instead of its current 7-9 grade junior high school. At this time th^ 
curriculum imbedded testing program does not extend past the sixth grade. 
Appropriate tests must be developed for the seventh and eighth grades in the 
new middle school configuration. 

High School . During the current school year (19B5-86) the high school 
will begin to develop formal course tests. This process faces several 
challenges. Traditionally, formative kinds of testing at the high school have 
been the exclusive domiiin of the teachers. A number of situations have arisen 
which make curriculum imbedded testing on a more formal basis necessary and 
desirable at the high school. The Oregon Essential Learning Skills program 
will carry with it stringent demands for highly visible accountability at the 
high school. High schools must.be in a position to react to the student's 
skills deficits and without improved assessment programs they will not be able 
to do that. Further, student mobility from district to district and school to 
school play an increasingly im^K>rtant role. As students from ot*ier programs 
enter local high schools it is increasingly iflQ)ortant that the high school be 
able to ascertain ^he student's level of attainment of these "essential 
skills" and be able to react to them. The traditional teacher constru::ted 
test program is not adequately designed for these purposes and standardised 
testing programs are not sufficiently sensitive fot these purposes. 
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The district states that curriculum imbedded testing at the high schrol 
level faces some fairly oLyious problems, not the least of which is the 
general attitude of high school tciichers. High school teachers tend to be 
much more subject matter oriented than skill oriented in their approach to 
instruction. Also, even within a given subject matter area, measurable course 
objectives are not consistent across te^ichers within that a ea. Curriculum 
imbedded tests inherently carry with them the concept of teach- reteach. This 
is particularly true when the results of testing have a nore public exposure 
than the results of the teacher constructed test. That other "publics" 
outside the classroom may .now how students perform on a particular test is a 
new concept for high school teachers end they tend to regard it as a threat. 

Another problem that the district sees in using this testing concept at 
the high school level has to do witii the sheer magnitude and variety of the 
courses taught. It is not economically possible to test all courses taught at 
al> levels, which courses will be tested? How will these decisions be made? 
leho will be involved in these decisions? All of these questions are now being 
addressed. At this time areas to be developed first are English, Math, and 
Science. The immediate goal is oriented toward "how to" as on>osed to formal 
testing per se. 

The district identifies a number of developmental areas which, vhen 
considered in total, present formidable cost concerns. High schocd teachers 
are not well trained in test development and construction. Staff development 
is a concern. Getting key people tr vthei to develop items and f;ests is an 
absolute necesaity. Released tim« :h j conc^ n. Securing and developing a 
scanning and scoring system must be don«. Technology and scftwara development 
^.s of concern. 
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Finally, a major concern for some high school teachers relates to the 
multiple-choice naturt^ of the machine- scored tests they are to develop. 
First, they feel that their course content is better measured through essay 
tests. Second, many courses atteii^pt to te&ch critical thinking, analysis and 
other higher order thinking skills which the teachers feel can't be measured 
in an objective test format. 
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APPENDIX A 
Number of Items on Each Test 
Reading Tests 



Number Number 

Test/ Course Number Performance Test 

Grade Goals Indicators (Skills) ittms 

1 13 40 380 

2 14 44 144 

3 13 49 181 

4 13 53 167 

5 13 51 171 

6 11 49 183 

Math Tests 

K 9 39 43 

1 12 34 50 

2 12 41 50 

3 12 54 100 

4 12 61 100 

5 12 62 100 

6 12 71 100 
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TESTING SCHEDULE 
FOR 1985-86 
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NOME 



HBJ - LFVEL 18/ PERlOPIC lESl 7 

hEMORIOL HURLEY GRADE 4 
IDNUhPf?R TOTOL- 2.1 2.2 - 2." — 2:4 



rwii f > t T D 



« DANIEL J 

A DRNIELLEE 

0 JASON 

A ELIZABETH R 
"JENNIFER"F — 

P JEFFREY H 

P MEL 1 NOP 

B KRISHNA 

C JEPNANN L 

C DEON 

"T: n^EPECCP^ 

H NICKY 

H KORY K 

""H "OAVID-V 

L KARRI 

O ERIC E 

■R t^TEVEN'P 

R MICHAEL K 

R SANDRA C 

S PATRICK^J-- 

T CATHERINE F 

V GEOFFREY K 

"2 PDAM 



PI0&f>2B 39 78y BZ% 7Z% 7B% 

00G027 ii4 " B9)t~92%"^ 91% 7E%' 

(n0S67CI f'09C 92% 100X 100% 

006660 50 100% 100% 100% 100% 

"00B02g - - Aft 96% ' R3% — 3\%—7Eir 

0057B6 38 &7% 97% 82% 67% 

0079G8 65 83% 83% 100% 100% 

006030 ' .39 ~ G7%~92%— B6% 100%' 

00R032 33 5G% 67% 91% 56% 

^7 577 33 33% 92% 10*^% 56% 

005033 37 56% — 83% — 91% — 78%" 

007972 68 100% 92% 100% 89% 

006057 32 56% 75% 73% 56% 

007760 - 38 72%""92%— 91%" 66%' 

007961 61 78% 92% 82% 78% 

00660! 67 96% 100% 100% 78% 

00E.5ie 6 3 83% 100%— F2%- 78%" 

006519 69 100% 100% 91% 100% 

006 P56 68 100% 92% 91% 100% 

006523 28 56% 50%"^:Tr — 66% 

006065 38 61% 83% b2% 89% 

006066 66 .96% 92% 91% 89% 
009fi91 ' 60 78%'- 92% — 82%— 67%^- 



TOTALS FOR THIS CLASS 60.9 76% 88% 8B% 78% 
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FORM USED TO CROSS-REFERENCE 
CURRICULUM OBJECTIVES TO TEXTBOOK OBJECTIVES 
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GRADE LEVEL GUIDE 



PROGRAM GRADE ITVEL 







COURSE GOAL 
^ Performance Indicator 


Textbook Reference 
and 

Other Resources 


Media 
Materials 


Assessment 
Procedures 


• 1 




10 




KSTCOPrWAHABLE 
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GRADE LEVEL GUIDE 



fmm Rfding 




COURSE GOAL 
PerforiMnce Indicator 



Tex^.book Reference 
and 

Other Resources 



Wr AO 
1-1 



LI AO 



E 
E 



E» 



THE STUDENT WILL: 

A) BE ABLE TO DEMONSTRATE PRE-READING SKILLS. 



AO 
AO 



provide evidence of namial dexterity, 
reproduce a printed form or pattern, 
write the letters of the alphabet 
without a andel. 



B) BE ABLE TO OEMONSTRATF AUDITORY 
DISCRININATION. 

B-3) select words that begin snd/or end with 

the same sound. 
BO) select words which contain the same 

vowel sounds. 
B-5| Identify rhyming sounds. 
6-6 Identify long and short voi#e1 sounds. 
B-7) respond appropriately to questions. 

THE STIOENT WILL: 

0 BE ABLE TO OCHTJNSTRATE VISUAL DISCRININATION. 

CO) distinguish likenesses and differences 
In pictures, symboU, shapes, sites and 
colors. 

C-6) recognlie upper and lower case letters. 



HBJ 1.2* 
H8J 1.20.3 
HBJ 1.2.3.3.4 



HBJ 1.4.2 

HBJ 2.2. 2.5 

HBJ 1.4.5, 2.4 
HBJ 2.2. 2.5 



HBJ 1.3. 1.5. 2.5 
HBJ l.S. 6.3 
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EX HI B IT A 



GRADE LEVEL 2nd Grade 



Harcourt Brace Jova>y>vl :h 
Unit References 



26 

4.7.10 

26 



16 

Level 5 
.9.15.20.24 



10 



.IS.l.B 
20.5 





1 H> 
1 ^ 






1 ^ 








1 t; 


S 


PERIO: 





P.2 

2.3.4| p.2 
6,7,8 



p. 3 
p.? 
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TEACHING-TESTING-TEACHING PROCESS 
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